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SYSTEM AND METHOD FOR TRANSMITTING PROGRAM DATA 
INCLUDING IMMERSIVE CONTENT 
Robert W. Nevins, Jr. 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to immersive 

content, and particularly to the manipulation and broadcast 
of immersive content . 

Description of the Related Art 

[0002] A program transmission system includes central 

equipment for receiving and distributing program signals to a 
plurality of viewer terminals (e.g. televisions, computers, 
etc.) . Figure 1 illustrates a known program transmission 
system 100 including program formatting unit 101, central 
equipment 103, and a cable system 107. Program formatting 
unit 101 typically assembles the program signals in digital 
form for transmission within program transmission system 100. 
Program formatting unit 101 can also generate information 
signals to accompany the program signals. Such information 
signals could include, for example, information regarding 
channel numbers, program titles, program lengths, and program 
start times. Central equipment 103 can receive program 
signals from program formatting unit 101 via a terrestrial 
link 111 or from a satellite 102 via a satellite link 110. 
[0003] Central equipment 103 can provide signal 

conversion 104, signal processing 105, and signal storage 
106. Specifically, the program signals received by central 
equipment 103 can be decoded or de-multiplexed (signal 
conversion 104) , as necessary, and then filed (signal storage 
106) for subsequent transmission over a cable system 107. 
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Viewer requests received via terminals 108 are transferred to 
cable system 107. Central equipment 103 interprets these 
requests (signal processing 105) and transfers the 
appropriate program signals to cable system 107 accordingly. 
[0004] Although program transmission system 100 can 

deliver conventional, digital cable programming, this system 
is ineffective in transmitting cutting edge program data, 
such as immersive content, for live broadcasts. Therefore, a 
need arises for a system and method of transmitting program 
data including immersive content. 



SUMMARY OF THE INVENTION 

[0005] The present invention provides a system and method 

for transmitting program data including immersive content. 
In accordance with one method, a generated immersive video 
stream is first choreographed. This choreography can be 
performed by an operator or using software. Then, based on 
this choreography, the relevant macroblocks of the immersive 
video stream can be identified and compressed. At this 
point, the compressed macroblocks of the immersive video 
stream are ready for transmission. 

[0006] In one embodiment, a plurality of immersive video 

streams can be choreographed and compressed. These 
compressed immersive video streams can be multiplexed for 
transmission and subsequently de-multiplexed for display. 
Before display, the relevant macroblocks can be decompressed. 
[0007] In accordance with one feature of the invention, a 

program transmission system includes means for receiving and 
choreographing the immersive content, a macroblock selection 
unit, wherein each selected macroblock corresponds to a 
relevant macroblock of the immersive content as determined by 
the means for receiving and choreographing, and a compression 
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unit to compress the selected macroblocks for transmission. 
The program transmission system typically includes a cable 
system that receives an output of the compression unit and 
provides at least one terminal for displaying the immersive 
content. The terminal can include a decompression unit to 
decompress the selected macroblocks. In one embodiment, the 
terminal can display a plurality of previews associated with 
at least one program, and at least one preview includes 
immersive content. The plurality of previews can be 
individually or simultaneously activated. 

[0008] In accordance with one feature of the present 

invention, an on-screen directory displayed at a terminal 
includes a plurality of preview areas and a plurality of 
corresponding textual descriptions. Each preview area can 
include a predetermined segment of a program, wherein the 
segment shows representative action or drama of the program. 
The plurality of preview areas can be simultaneously or 
individually active. In the embodiment in which the preview 
areas are simultaneously active, one or more guides can 
indicate how to select a program or return to another 
directory. In the embodiment in which the preview areas are 
individually active, one or more guides can indicate how to 
preview a program, i.e. activate a preview area, select that 
program, or return to another directory. 

[0009] In another embodiment of the invention, an on- 

screen display includes a plurality of preview areas and a 
selected program area. Typically, the selected program area 
is larger than each of the plurality of preview areas. In 
this embodiment, the plurality of preview areas can be 
individually or simultaneously active. In the embodiment in 
which the preview areas are simultaneously active, one or 
more guides can indicate how to select a program or return to 
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another directory. In the embodiment in which the preview 
areas are individually active, one or more guides can 
indicate how to preview a program, i.e. activate a preview 
area, select that program, or return to another directory. 
[0010] The on-screen directory including previews 

provides significant advantages over the known art. 
Specifically, a preview provides significantly more 
information than a mere listing of channels and their 
designations. Moreover, unlike those listings relying on 
written descriptions, a preview provides unbiased information 
to the viewer. Thus, the on-screen directory of the present 
invention, by showing representative action or drama of the 
program, allows the viewer to quickly identify a program of 
interest . 

[0011] In accordance with one feature of the invention, 

at least one of the program can include immersive content, 
wherein the immersive content allows the location of a viewer 
within an environment to be moved. The viewer viewing 
immersive content can experience an event from any angle as 
if experiencing the event firsthand. For example, in a 
driving video, the viewer could have any angle as if the 
viewer were in the driver's seat of a car, thereby allowing 
the viewer to look left, right, ahead, above, and in the rear 
view mirror, as desired. 

BRIEF DESCRIPTION OF THE FIGURES 

[0012] Figure 1 illustrates a block diagram of a 

conventional program delivery system. 

[0013] Figure 2 is a three-dimensional representation of 

a viewer and an environment . 

[0014] Figure 3 is a three-dimensional representation for 

texture mapping a spherical environment on a cube. 
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[0015] Figure 4 is a three-dimensional representation for 

texture mapping a spherical environment on a cylinder. 
[0016] Figure 5 illustrates a block diagram of a program 

delivery system in accordance with the present invention. 
[0017] Figure 6 is a diagram of a digital image separated 

into macroblocks. 

[0018] Figure 7 is a block diagram of a macroblock 

selection unit. 

[0019] Figure 8 is a block diagram of a video stream 

compressor . 

[0020] Figure 9 is a block diagram of a video stream 

decompression unit. 

[0021] Figure 10A illustrates one embodiment of an on- 

screen directory in accordance with the present invention. 
[0022] Figure 10B illustrates the on-screen directory of 

Figure 10A in which previews are simultaneously active. 
[0023] Figure 10C illustrates the on-screen directory of 

Figure 10A in which previews are individually active. 
[0024] Figure 11A illustrates another embodiment of an 

on-screen directory in accordance with the present invention. 
[0025] Figure 11B illustrates the on-screen directory of 

Figure 11A in which previews are simultaneously active. 
[0026] Figure 11C illustrates the on-screen directory of 

Figure 11A in which previews are individually active. 

DETAILED DESCRIPTION OF THE FIGURES 

Immersive Content 
[0027] Program data can include immersive content to 

provide a dynamic environment for a viewer. For example, an 
immersive video can be made to capture a safari in the 
Serengeti, a car drive down Highway 1 in California, a 
concert with Madonna, or a plane ride through the Grand 
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Canyon. Immersive video techniques provide a dynamic 
environment by allowing the location and the view window of 
the viewer to be simultaneously moved. Thus, in the 
California Highway 1 immersive video, the viewer could have 
the perspective of being in the driver's seat of a speeding 
Ferrari and look at the ocean, the sandstone cliffs, or the 
redwood forests, as desired. 

[0028] Immersive content is generated by providing 

multiple environment maps of three-dimensional spaces. 
Figure 2 illustrates the construct used in one environment 
mapping system. A viewer 2 05 (represented by an angle with a 
curve across the angle) is centered at the origin of a three 
dimensional space having X, Y, and Z coordinates. The 
environment of viewer 2 05 (i.e. what the viewer can see) is 
ideally represented by a sphere 210, which surrounds viewer 
205. Generally, for ease of calculation, sphere 210 is 
defined with a radius of 1 and is centered at the origin of 
the three dimensional space. More specifically, the 
environment of viewer 2 05 is captured and then re-projected 
onto the inner surface of sphere 210. Viewer 205 has a view 
window 23 0 which defines the amount of sphere 210 that viewer 
205 can see at any given moment. View window 230 is 
typically displayed on a display unit for the viewer of the 
environment mapping system. 

[002 9] Computer graphic systems are generally not 

designed to process and display spherical surfaces. Thus, as 
illustrated in Figure 3, texture mapping techniques are used 
to create a texture projection of the inner surface of sphere 
210 onto polygonal surfaces of a regular solid (i.e. a 
platonic solid) having sides that are tangent to sphere 210. 
As illustrated in Figure 3, a common texture projection is a 
cube 320 surrounding sphere 210. Specifically, the 
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environment image on the inner surface of sphere 210 serves 
as a texture map, which is then texture mapped onto the inner 
surfaces of cube 320. A cube is typically used because most 
graphics systems are optimized to use rectangular displays 
and a cube provides six rectangular faces. The faces of the 
cube can be concatenated together to form the environment 
map. During viewing, the portions of the environment map 
that correspond to view window 23 0 (Figures 2 and 3) are 
displayed for viewer 205. 

[0030] Other texture projections can also be used. For 

example, cylindrical mapping, as illustrated in Figure 4, can 
be used if view window 23 0 is limited to a visible range 
around the equator. Specifically, in Figure 4, a texture 
projection in the shape of a cylinder 42 0 surrounds sphere 
210. The environment image on the inner surface of sphere 
210 serves as a texture map, which is then texture mapped 
onto the inner surface of cylinder 420. Note that cylinder 
420 can be approximated using a plurality of rectangular 
sides to simplify the texture mapping. Moreover, cylinder 
420 may be "unrolled" to form a rectangular environment map. 
Texture mapping is described in further detail in the 
reference, "Texture Mapping as a Fundamental Drawing 
Primitive", published in June, 1993 by Paul Haeberli and Mark 
Segal . 

[0031] An immersive video stream comprises a series of 

individual digital images (also called frames) , wherein each 
digital image is an environment map. For full motion 
immersive video, a video frame rate of 3 0 images per second 
is desired. A digital image comprises a plurality of picture 
elements (pixels) , wherein each pixel can be identified using 
a 2 dimensional coordinate system. Typical image sizes for 
conventional digital video streams include 640x480, 320x240 
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and 160x120 pixels. However, image sizes for immersive video 
streams (i.e. environment map sizes) are typically much 
larger than conventional digital video streams. Some common 
image sizes for immersive video streams include 1024x1024 and 
2048x2048 pixels. 

Immersive Content Transmission System 
[0032] Program transmission system 100 is unable to 

interpret and manipulate immersive program data for live 
broadcasting. Specifically, the complexity and size of the 
multiple environment maps comprising the immersive program 
data render conventional techniques and components used in 
program transmission system 100 ineffective. To address 
these issues, the present invention provides a system and 
method for transmitting program data including immersive 
content . 

[0033] Figure 5 illustrates a block diagram of an 

immersive content transmission system 500 that includes 
immersive content creation unit 501, central equipment 503, 
and a cable system 508 having at least one terminal 509. 
Immersive content creation unit 501 generates a digital video 
stream including multiple environment maps. Immersive 
content creation unit 501 can also generate information 
signals to accompany the digital video stream. Such 
information signals could include, for example, information 
regarding a channel number and a program title. Central 
equipment 503 can receive the immersive content from 
immersive content creation unit 501 via a terrestrial link 
511 or from a satellite 502 via a satellite link 510. 

Signal Choreographing 
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[0034] In accordance with the present invention, central 

equipment 503 can provide signal choreographing 504. During 
choreographing, the multiple environment maps are analyzed 
and at least one view window in each environment map is 
defined for transmission. This choreography can be performed 
by an operator or defined by software. In the case of 
operator input, the operator can use a standard viewer input 
device (e.g. a joy stick) to determine the view window. In 
the case of software definition, the software could begin 
with a predetermined view window, such as the view window of 
a driver looking at the road, to place the viewer in the 
context of the immersive video. Then, the view window could 
be changed according to predetermined parameters. These 
parameters could be based on viewer preferences (as 
determined by surveys, tests, or any other known methodology) 
or set to simulate a more realistic immersive experience. 
Thus, assuming that the immersive content simulated a car 
ride down Highway 1 in a Ferrari, the operator or software 
would determine what the viewers at terminals 509 would see. 

Compression Schemes 
[0035] Because digital video streams, especially 

immersive video streams, contain considerable data, central 
equipment 503 can use signal compression 505 to reduce data 
storage and transfer requirements. Typical compression 
schemes define macroblocks, wherein a macroblock includes a 
plurality of adjacent pixels with a predetermined shape and 
size. A typical size for a macroblock is 16 pixels by 16 
pixels. Figure 6 illustrates a digital image 610 divided 
into a plurality of square macroblocks MB. In Figure 6, 
digital image 610 has X columns and Y rows of macroblocks MB, 
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thereby allowing each macroblock MB to be identified using a 
2 dimensional coordinate system. 

[0036] Various compression schemes can be used to 

compress digital video streams. One common technique for 
compressing a digital video stream is to transmit a 
difference frame, which contains data specifying the 
difference between previous or following frames rather than 
transferring each image of the video stream. For example, P 
frames and B frames used in MPEG compression would be 
classified as difference frames. Periodically, a self- 
contained frame can be transmitted so that errors caused by 
sending only difference frames do not accumulate beyond a 
reasonable level. As used herein, a self-contained frame 
contains all the data necessary to be fully decoded into an 
image. For example, an I frame (interframe) in MPEG 
compression is a self-contained frame as used herein. In 
MPEG compression, an I frame is typically transmitted every 
twelfth frame. However, in some embodiments of the present 
invention, a self-contained frame may be sent more often, 
such as once every fifth frame. 

[0037] Generally, differences are calculated on a 

macroblock level. Thus, a difference frame contains 
difference macroblocks. However, some macroblocks within 
difference frames may be self-contained macroblocks. During 
encoding the macroblocks are first converted from a display 
color space such as RGB into a luminance based color space. 
In general, luminance based color spaces, such as YC b C r , are 
used because the luminance signal provides most of the 
information needed for compression. 

[0038] A difference macroblock can be calculated based on 

other macroblocks contained within the current frame or 
within frames in close temporal proximity with the current 
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frame. Typically, a frame range FR is specified during 
encoding. For example, if the frame range FR is two frames, 
then the current frame and the two frames following the 
current frame can be used to encode the difference 
macroblock. In some compression schemes, instead of 
specifying a frame range FR, the frame range FR can be 
determined by the number and spacing of self-contained frames 
within the video stream. For these embodiments a maximum 
frame range can be determined for the video stream and used 
in place of the specified frame range. 

[0039] In addition to temporal proximity, a difference 

macroblock can also be calculated based on macroblocks in 
close area proximity to the current macroblock. Typically, 
this area range AR is specified during encoding to determine 
the necessary level of area proximity. Various other 
schemes, such as motion estimation and discrete cosine 
transform coding, can also be used with macroblocks to 
further compress the video stream. 

[0040] Because immersive video streams are significantly 

larger and contain far more data than standard video streams, 
compression (and subsequent decompression) of high resolution 
immersive video streams may be beyond central equipment 103 
(Figure 1) as well as the video display systems in terminals 
108 designed for standard video streams. Thus, in accordance 
with one embodiment of the present invention, only the 
portion of the immersive video stream in view window 230 
(Figure 2) as determined by signal choreographing 504 (Figure 
5) is fully compressed. However, macroblocks near the edge 
of view window 23 0 may depend on macroblocks outside of view 
window 230. Thus, macroblocks near view window 230 may need 
to be partially or fully compressed in accordance with other 
embodiments of the present invention. As used herein, the 
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term "relevant" macroblock refers to members of the subset of 
macroblocks that are necessary for creating the image in view 
window 230 as determined signal choreographing 504. 
[0041] Figure 7 is a block diagram of one embodiment of a 

macroblock selection unit 700 which can be used select a 
subset of selected macroblocks. In macroblock selection unit 
700, a view frustum calculation unit 710 receives information 
regarding view window 230, view window motion parameters 702, 
and encoding parameters 704 to calculate an expanded view 
window for each frame. This expanded view window encompasses 
the relevant macroblocks within each frame. 
[0042] View window motion parameters 702 include a 

maximum horizontal speed HS max and a maximum vertical speed 
VS max for view window 730. Generally, maximum horizontal 
speed HS m ax and maximum vertical speed VS max are defined using 
the unit pixels/frame. Thus, if maximum vertical speed VS max 
is equal to 5 pixels/frame, view window 230 can move up or 
down at a rate of 5 pixels per frame. In some embodiments, 
rather than using the speed of view window 23 0 in terms of 
pixels per frame, spatial parameters based on the viewpoint 
of viewer 2 05 is used. For example, parameters describing 
the angular rate at which the view frustum moves can be used 
in place of the speed of the view window. Encoding 
parameters 704 include the above -referenced frame range FR 
and area range AR. U.S. Patent Application 09/670,957, also 
filed by the assignee of the present invention, describes 
several exemplary definitions for frame range FR and area 
range AR in further detail. 

[0043] After calculating the expanded view window, view 

frustum calculator 710 calculates the normal vectors of an 
expanded view frustum encompassing the expanded view window. 
A view frustum is the solid angle projection from viewer 205 
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(who is deemed to be at the origin) , which encompasses the 
expanded view window. Generally, the expanded view window is 
rectangular, thus the expanded view frustum for the expanded 
view window would resemble a four- sided pyramid and have four 
normal vectors, i.e. one vector for each side of the expanded 
view frustum. In one embodiment, a view frustum normal 
vector points into and perpendicular to the plane containing 
a side of the expanded view frustum. 

[0044] A macroblock vertex classifier 730 receives the 

calculated view frustum normal vectors as well as macroblock 
coordinates 705. A coordinate conversion unit 720 can 
convert the macroblock coordinates from environment map 
coordinates to spatial coordinates around viewer 205. Note 
that the coordinate system of the expanded view window can be 
the same as the coordinate system of the macroblocks, thereby 
rendering coordinate conversion unit 72 0 unnecessary. 
[0045] Macroblock vertex classifier 730 uses the view 

frustum normal vectors to classify each vertex of every 
macroblock (four vertices for each macroblock MB in the 
embodiment shown in Figure 6) to determine whether that 
vertex is above, below, left of, or right of the view 
frustum. The relationship of a vertex with the view frustum 
is computed using the inner product (or dot product) of the 
vertex with the view frustum normal vectors. For example, if 
the inner product of a vertex with the right side view 
frustum normal vector is less than zero, then the vertex is 
to the right of the view frustum. Similarly, if the inner 
product of a vertex with the left side view frustum normal 
vector is less than zero, then the vertex is to the left of 
the view frustum. 

[0046] Macroblock vertex classifier 730 provides the 

vertex classifications to a macroblock selector 740. 
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Macroblock selector 740 then selects a subset of selected 
macroblocks 745, wherein the subset includes any macroblock 
that has at least one vertex within the view frustum. In one 
embodiment, due to processing efficiencies, subset of 
selected macroblocks 745 can include some macroblocks other 
than relevant macroblocks. U.S. Patent Application 
09/670,957, referenced herein, describes an exemplary 
macroblock selector 740. 

[0047] Once the relevant macroblocks are identified, the 

choreographed immersive program data can be compressed. 
Figure 8 is a simplified block diagram of a video stream 
compressor 800 that receives input from macroblock selection 
unit 700. Video stream compressor 800 includes a color 
converter 810, a difference calculator 820, a discrete cosine 
transform (DCT) calculator 830, and a variable length encoder 
840. Color converter 810 converts the choreographed 
immersive video stream CIVS into a luminance based video 
stream LVS . Specifically, color converter 810 converts the 
color space of each macroblock in choreographed immerisve 
video stream CIVS into a compressed macroblock color space, 
typically luminance based. Difference calculator 820 
converts luminance -based video stream LVS into a difference 
video stream DVS, which has both self-contained frames and 
difference frames. Specifically, difference calculator 680 
encodes a subset of the relevant macroblocks in luminance- 
based video stream LVS into difference macroblocks. DCT 
calculator 83 0 performs a discrete cosine transform on 
difference video stream DVS to create a discrete cosine 
transform video stream DCTVS. Specifically, each relevant 
macroblock is transformed into a DCT macroblock comprising 
DCT coefficients. Compression is achieved using DCT 
transforms by quantizing the DCT coefficients. Finally, 

14 



ERT-021 PATENT 

variable length encoder 840 performs a final encoding to 
create a compressed video stream CVS. Specifically, variable 
length encoder 840 assigns different bit patterns to the DCT 
coefficient values of the DCT macroblocks . Compression is 
achieved by using shorter patterns for more common DCT 
coefficient values. Note that run length encoding can also 
be applied to the relevant macroblocks to further compress 
the video stream. Note that for illustration purposes, a 
compression scheme conforming to MPEG has been described. 
However, one skilled in the art can adapt the principles of 
the present invention to be used with a variety of 
compression schemes. For example, in accordance with another 
embodiment of the present invention, the compression standard 
called Digital Video (DV) can also be used. DV compression, 
in contrast to MPEG compression that reduces redundancy from 
frame -to -frame, reduces redundancy only within one frame. DV 
compression is explained in detail in U.S. Patent 6,233,282, 
which issued to Adaptec, Inc. on May 15, 2 001, and U.S. 
Patent 6,215,909, which issued to Sony Electronics, Inc. on 
April 10, 2001. 

[0048] After signal compression 505 (Figure 5) , central 

equipment 503 can provide immediate signal processing 507 of 
the immersive video stream or signal storage 506 of the 
immersive video stream for subsequent distribution over cable 
system 508. Specifically, viewer requests generated by 
viewer input devices 512 and provided to terminals 509 are 
thereafter transferred to cable system 508 and then forwarded 
to central equipment 503. Central equipment 503 can 
interpret these requests (signal processing 507) and can 
transfer the appropriate program signals to cable system 508 
accordingly. 
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[0049] A viewer request can be initiated using any 

standard viewer input device 512. For example, if terminal 
509 is a television, then viewer input device 512 could 
include a remote control or a control panel . Such viewer 
input devices typically include buttons labeled with alpha, 
iconic, and numeric characters as well as movement cursors. 
If terminal 509 is a computer, then viewer input device 512 
could include a mouse having the ability to move a cursor and 
select objects on-screen. Other types of viewer input 
devices 512, such as joysticks, can be used with various 
terminals 509. 



Decompression of Immersive Content 
[0050] Figure 9 is a block diagram of a video stream 

decompression unit 900 for incorporation into a terminal 509 
in accordance to one embodiment of the present invention. In 
this embodiment, decompression unit 900 includes a variable 
length decoder 910, an inverse DCT calculator 92 0, a frame 
restorer 930, and a color converter 940. Variable length 
decoder 910 receives the compressed video stream CVS and 
decodes the variable length encoding performed by variable 
length encoder 840 (Figure 8) to generate a restored discrete 
cosine transform video stream RDCTVS . Specifically, run 
length decoding is used to extract the DCT coefficients bit 
patterns of the relevant macroblocks and the bit patterns are 
converted to actual DCT coefficients. Inverse discrete 
cosine transform (DCT) calculator 920 receives restored 
discrete cosine transform video signal RDCTVS and generates a 
restored difference video stream RDVS by inverting the 
discrete cosine transform performed by DCT calculator 830 
(Figure 8) on the relevant macroblocks. A frame restorer 930 
receives the restored difference video stream RDVS and 
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generates a restored luminance based video stream RLVS . 
Specifically, frame restorer 93 0 reforms the relevant 
macroblocks from the difference macroblocks of restored 
difference video stream RDVS . 

[0051] Finally, a color converter 940 converts the color 

space of the relevant macroblocks in restored luminance-based 
video stream to a color space suitable for the display 
systems of terminals 509 (Figure 5) . For example, in one 
embodiment of the present invention, the color space of the 
compressed macroblocks is 4:2:0 YCbCr and the color space for 
the display system is 4:4:4 RGB. Other luminance -based color 
spaces that may be used include 4:2:2 YCbCr and 4:4:4 YCbCr. 
In this manner, video stream decompression unit 90 0 outputs a 
restored, choreographed immersive video stream RCIVS to 
terminals 509. 



On-Screen Directories 
[0052] As described above, terminals 509 can include 

televisions, computers, or other viewer display systems. For 
illustration purposes, the following descriptions of various 
embodiments of on-screen directories assume that a terminal 
509 includes a television and that the associated viewer 
input device 512 of that terminal 509 includes a remote 
control having at least numerical buttons. However, those 
skilled in the art can adapt the principles of the present 
invention to be used with a variety of types of terminals 
and/or associated viewer input devices. 

[0053] On-screen directories for terminals 509 are known 

in the art. In one directory familiar to most viewers, a 
designated channel provides a comprehensive list of channels 
and their corresponding designations (e.g. NBC, The Discovery 
Channel, CNN, etc.). This directory is useful to efficiently 
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direct the viewers to a desired channel, but provides no 
information to the viewers regarding the programs showing on 
that channel. In another directory provided by Tivo, which 
provides a personal video recorder service, a screen provides 
a synopsis of each program provided by a channel. This 
directory further includes a still shot taken from the 
program, typically of one of the principal actors or action 
scenes. Thus, for the viewer to determine whether the 
program is of interest, the viewer must read the synopsis. 
This process can be tedious and subject to bias (as each 
synopsis can be written to seem interesting) . Therefore, 
each of the above directories, whether providing no 
information or biased information regarding a program, fails 
to give the viewer the necessary tools to determine whether a 
particular program will be of interest to the viewer. 
[0054] Figure 10A illustrates one embodiment of an on- 

screen directory 1000 including a plurality of preview areas 
1001A, 1001B, 1001C and 1001C, wherein each preview area 1001 
has a corresponding textual description 1002. Specifically, 
preview area 1001A has a corresponding textual description 
1002A, preview area 1001B has a corresponding textual 
description 1002B, preview area 1001C has a corresponding 
textual description 1002C, and preview area 1001D has a 
corresponding textual description 1002D. Each preview area 
1001 can include a predetermined segment of a program, 
wherein the segment shows representative action or drama of 
the program. The plurality of preview areas 1001 can be 
simultaneously or individually active. 

[0055] In the embodiment in which preview areas 1001A- 

1001D are simultaneously active, a viewer can select a 
program (i.e. run the program on another screen) by entering 
a corresponding number of the preview area 1001 via a viewer 
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input device. For example, referring to Figure 10B, preview 
area 1001A can be selected by pressing number 1 on the viewer 
input device (as indicated by guide 1003A) , preview area 
1001B can be selected by pressing number 2 on the viewer 
input device (as indicated by guide 1003B) , preview area 
1001C can be selected by pressing number 3 on the viewer 
input device (as indicated by guide 1003C) , and preview area 
1001D can be selected by pressing number 4 on the viewer 
input device (as indicated by guide 1003D) . The viewer can 
return to another directory, such as the listing of 
television channels, by pressing number 0 on the viewer input 
device (guide 1004) . 

[0056] In the embodiment in which preview areas 1001A- 

1001D are individually active, a viewer can preview a program 
(i.e. activate a preview area 1001) by entering a first 
number via a viewer input device and select that program by 
entering a second number via the viewer input device. For 
example, referring to Figure 10C, a viewer can preview a 
program described by description 1002A by pressing number 1 
and can select that program by pressing number 5 on the 
viewer input device (as indicated by guide 1005A) . The 
viewer can preview a program described by description 1002B 
by pressing number 2 and can select that program by pressing 
number 6 on the viewer input device (as indicated by guide 
1005B) . A viewer can preview a program described by 
description 1002C by pressing number 3 and can select that 
program by pressing number 7 on the viewer input device (as 
indicated by guide 1005C) . Finally, the viewer can preview a 
program described by description 1002D by pressing number 4 
and can select that program by pressing number 8 on the 
viewer input device (as indicated by guide 1005D) . Once 
again, the viewer can return to another directory, such as 
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the listing of television channels, by pressing number 0 on 
the viewer input device (as indicated by guide 1004) . 
[0057] In another embodiment of the invention, an on- 

screen display includes a plurality of preview areas and a 
selected program area. For example, Figure 11A illustrates 
an on-screen display 200 including a plurality of preview 
areas 1101A-1101D and a selected program area 1102. In this 
embodiment, the plurality of preview areas 1101 can be 
individually or simultaneously active. 

[0058] In the embodiment in which preview areas 1101A- 

1101D are simultaneously active, a viewer can select a 
program (i.e. run the selected program in selected program 
area 1102) by entering a corresponding number of the preview 
area 1101 via a viewer input device. For example, referring 
to Figure 11B, preview area 1101A can be selected by pressing 
number 1 on the viewer input device (indicated by guide 
1103A) , preview area 1101B can be selected by pressing number 
2 on the viewer input device (indicated by guide 1103B) , 
preview area 110 1C can be selected by pressing number 3 on 
the viewer input device (indicated by guide 1103C) , and 
preview area 1101D can be selected by pressing number 4 on 
the viewer input device (indicated by guide 1103D) . The 
viewer can return to another directory, such as the listing 
of channels, by pressing number 0 on the viewer input device 
(indicated by guide 1104) . 

[0059] In the embodiment in which preview areas 1101A- 

1101D are individually active, a viewer can preview a program 
(i.e. activate a preview area 1101) by entering a first 
number via a viewer input device and select that program 
(i.e. run the program in selected program area 1102) by 
entering a second number via the viewer input device. For 
example, referring to Figure 11C, a viewer can preview a 
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program associated with a still shot shown in preview area 
1101A by pressing number 1 and can select that program by 
pressing number 5 on the viewer input device (indicated by 
guide 1105A) - The viewer can preview a program associated 
with a still shot shown in preview area 1101B by pressing 
number 2 and can select that program by pressing number 6 on 
the viewer input device (indicated by guide 1105B) . A viewer 
can preview a program associated with a still shot shown in 
preview area 1101C by pressing number 3 and can select that 
program by pressing number 7 on the viewer input device 
(indicated by guide 1105C) . Finally, the viewer can preview 
a program associated with a still shot shown in preview area 
110 ID by pressing number 4 and can select that program by 
pressing number 8 on the viewer input device (indicated by 
guide 1105D) . Once again, the viewer can return to another 
directory, such as the listing of television channels, by 
pressing number 0 on the viewer input device (indicated by 
guide 1104) . 

[0060] Typically, selected program area 1102 is larger 

than each of the plurality of preview areas 1101. In one 
embodiment (not shown) , using the viewer input device, 
selected preview area 1102 can be further enlarged to 
encompass an entire screen. 

[0061] The on-screen directories described in the 

embodiments illustrated in Figures 10A-10C and 11A-11C 
provide significant advantages over the known art. 
Specifically, a preview provides significantly more 
information than a mere listing of channels and their 
designations. Moreover, unlike those listings relying on 
written descriptions, a preview provides unbiased information 
to the viewer. Thus, the on-screen directory of the present 
invention, by showing representative action or drama of the 
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program, allows the viewer to quickly identify a program of 
interest . 

On-Screen Directories Including Immersive Content 
[0062] In accordance with one feature of the invention, 

one or more programs can include immersive content, as 
described in detail above. In one embodiment, the previews 
can include various aspects of the same "event" (thus, as 
used herein, the term "program" can refer to an identifiable 
aspect of any event) . For example, assuming an event is a 
live broadcast of a football game, each of the previews could 
focus on a specific part of the game, such as the defensive 
line of team A, the offensive play of team B, crowd reaction, 
etc. In this embodiment, immersive content creation unit 501 
(Figure 5) could capture the multiple environment maps using 
multiple cameras and central equipment 503 could choreograph 
and compress the incoming immersive data streams in parallel. 
For example, multiple operators could be used to choreograph 
the incoming immersive video streams, wherein one operator 
could determine what the viewers see regarding that specific 
aspect of the event. In one embodiment, signal processing 
507 can include multiplexing the plurality of choreographed 
immersive video streams for transmission to cable system 508. 
In this case, cable system 508 could include known components 
(not shown) for transmitting each choreographed immersive 
video stream as a preview. Note that because of their 
digital format the plurality of choreographed immersive video 
streams can be transmitted on a single cable television 
channel, telephone line, etc. 

[0063] Although the present invention has been described 

in detail with respect to certain embodiments, those skilled 
in the art will recognize modifications and variations that 
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are within the scope of the present invention. For example, 
although the above-described on-screen directories activate 
previews or selecto program using numeric characters on a 
viewer input device, other embodiments of the invention can 
use other means for activation/selection. Specifically, such 
means can include, but is not limited to, the use of any 
button, key, cursor, or menu provided on or in association 
with the viewer input device. Therefore, the present 
invention is defined only by the appended claims. 
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